1. Field of Invention
The invention relates generally to the fields of content delivery over a network, and in one exemplary aspect to improving the operation of a content-based network by using bandwidth-related information associated with input content to, inter alia, provide bitrate-based multiplexing of program streams.
2. Description of Related Technology
Video and/or audio programs are often encoded and compressed using hybrid discrete cosine transform (DCT) or differential pulse coded modulation (DPCM) encoding techniques such as those associated with well-known Motion Pictures Experts Group (MPEG), Windows Media 9/10 (WM 9/10), and similar encoding formats. Hybrid DCT/DPCM encoders (and other types of encoders) produce encoded output bits at a rate which varies with time. For example, depending on the motion and texture in the source material, the instantaneous output of an unbuffered MPEG encoder may vary between 1 and 6 MB/s for broadcast quality compression of standard definition video input material.
In many digital video encoding and distribution systems, the variable bit rate (VBR) output of an encoder is buffered and converted to a constant bitrate (CBR) stream. This is achieved by, e.g., reducing the instantaneous bitrate when it exceeds a target rate, and by adding null bits or otherwise “stuffing” the bit stream when the bitrate falls below the target rate. This “conditioning” of the stream is typically performed by or after each encoder, but before multiplexing the program streams into a common stream suitable for conveyance on a service provider's transmission network.
In a conventional digital television network, digital programming thus encoded is usually collected at a central location, assembled in multiple transport streams and transported to other intermediate locations in the network for further downstream transport to consumer premises equipment (CPE). These intermediate locations may alter the aggregated program multiplex received from the central location by replacing or changing the rate associated with programs in the original multiplex. For example, in a coaxial cable network, a cable operator can choose to insert local programming or advertisements at a hub location, and/or modify the aggregated programs received from a headend before the signal is sent downstream to the subscriber premises or other distribution nodes. The replacement programming may be available at the intermediate location in uncompressed (e.g., analog) or pre-compressed (e.g., MPEG-2 transport stream) format. In recent years, more and more central locations (e.g., cable headends) have begun using statistical multiplexing techniques to efficiently create the centrally aggregated program multiplexes.
As is well known, statistical multiplexing (colloquially referred to as “stat-mux”) is a technique used to efficiently pack multiple programs within a transport stream. This technique relies on the principle that the instantaneous bandwidth required to transmit a given program fluctuates over time, typically based on the ease of compression of the video content. This makes bandwidth-efficient transmission of multiple programs possible as a multiplex by sharing the total allocated bitrate. Because the bitrate peaks of separate program streams do not occur simultaneously, a group of programs can share an allocated bitrate that is smaller than the sum of the bitrate peaks of the program streams carried. Furthermore, the bitrate contribution each individual program stream is typically controlled (commonly referred to as “rate shaping”) to provide both a safety factor or margin, and even greater overall bitrate efficiencies. Conventional approaches to statistical multiplexing have recognized that the greater the number of programs in a multiplex (i.e., “pool size”), the better the chances of using bandwidth efficiently, due in part to a more even statistical distribution of instantaneous bitrates.
While the use of statistical multiplexing is beneficial in many aspects, including for example reduced storage and transport costs, it creates a new set of design challenges that must be adequately addressed. For example, when the program streams sent from the headend to a hub are statistically multiplexed together, insertion of new material at the hub presents additional considerations and challenges for maximization of the multiplex efficiency. When a statistically multiplexed digital program or content is removed from the multiplex, and another digital program is inserted in its place, the instantaneous bandwidth requirements of the two programs are almost never identical. If the instantaneous bandwidth of the original program is less than the instantaneous bandwidth required by the program to be inserted, the program to be inserted has to be quantized to a lower rate, resulting in a loss in quality. However, when the instantaneous bandwidth of the original program is greater than that of the program to be inserted, the quality of a previously encoded replacement program cannot be improved any more than what is already present. Therefore, replacement of a statistically multiplexed program often results in a reduction in quality of the inserted program. In some conventional systems, this quality problem is addressed by providing unused bandwidth in the original multiplex to accommodate for the differences in instantaneous bandwidths.
A variety of approaches to statistical multiplexing and communication between encoding stages are in evidence under the prior art. For example, U.S. Pat. No. 5,708,664 to Budge, et al. issued Jan. 13, 1998 entitled “Statistical multiplexing” discloses a transmitter for transmitting a plurality of digital signals through a plurality of channels, the channels having a predetermined total allocated bitrate. The transmitter includes a plurality of encoders each associated with one channel; a multiplexer for receiving the encoded digital signals and for transmitting the encoded signals as a stream of data, and operable for adjusting the distribution of the bitrate allocation between and among the encoded signals, and a processing device for providing an indication of a target quality and an actual quality for each channel and for causing the multiplexer to repeatedly adjust the distribution of the bitrate allocation in response to differences between the indicated actual quality and the indicated target quality for each channel so as to equalize differences between the actual and target quality across at least some of the channels. By grouping encoders together in “statistical multiplex groups”, and making real time decisions about the bitrate requirements for those encoders, bitrate can be allocated to maximize picture quality for the group. For a variety of different picture sources in a statistical multiplex group, to achieve a target picture quality the bitrate requirements of each will vary with coding difficulty. Thus, a channel within the statistical multiplex group that is experiencing little difficulty in encoding its picture can free bits to channels that are having greater difficulty. The effect is to smooth the picture quality and subjectively improve it.
U.S. Pat. No. 6,219,358 to Pinder, et al. issued Apr. 17, 2001 entitled “Adaptive rate control for insertion of data into arbitrary bit rate data streams” discloses an apparatus wherein the rate of insertion of data, such as MPEG table packets, into an outgoing bit stream is varied by a packet handler. The packet handler, which is located in a modulator in a cable television system headend, includes control logic and a packet router. The actual insertion rate of the outgoing data is based on the bit stream's available capacity for insertion of data and the desired insertion rate of the data. When the available capacity for insertion equals or exceeds the desired insertion rate, the actual insertion rate equals the desired insertion rate. When the available capacity for insertion is less than the desired insertion rate, the actual insertion rate is reduced from the desired insertion rate. The invention dynamically determines the available capacity for insertion and adjusts the actual insertion rate.
U.S. Pat. No. 6,285,716 to Knee, et al. issued Sep. 4, 2001 entitled “Video compression” discloses a method to manipulate an MPEG-2 or other compressed video stream as separate information bus and coefficient streams. The information bus stream contains motion vector information but also information derived from a previous decoding operation for use in a subsequent coding operation. Processing in the coefficient domain enables bit rate conversion without decoding to the pixel level and also ostensibly simplifies the combination of MPEG layers.
U.S. Pat. No. 6,577,682 to Brightwell, et al. issued Jun. 10, 2003 entitled “Video processing system also compressing coding decision data” discloses a method in which an MPEG2 decoded video signal is accompanied by a representation of the coding decisions to aid downstream re-encoding. The representation is MPEG compliant bit modified as an attempt to reduce the number of bits.
U.S. Pat. No. 6,792,045 to Matsumura, et al. issued Sep. 14, 2004 entitled “Image signal transcoder capable of bit stream transformation suppressing deterioration of picture quality” discloses how an MPEG2 decoder portion decodes an input bit stream and outputs a digital decoded image while extracting coding information and supplying the same to a control portion. An MPEG2 encoder portion re-encodes the digital decoded image output from the MPEG2 decoder portion. Coding information supplied from the control portion is reflected on a coding parameter in re-encoding. Transcoding between the MPEG2 standard and the DV standard can also be executed by arranging a decoder or an encoder corresponding to the DV standard in place of either the MPEG2 decoder portion or the MPEG2 encoder portion.
U.S. Pat. No. 6,795,506 to Zhang, et al. issued Sep. 21, 2004 entitled “Methods and apparatus for efficient scheduling and multiplexing” discloses techniques and mechanisms for scheduling and multiplexing compressed bitstreams. A compressed bitstream includes bit rate information describing the bit rate of video data. The bit rate information is used to ostensibly improve the scheduling and multiplexing efficiency of compressed bitstreams. Compressed video data can be transmitted over communication channels at bit rates that comply with available channel bandwidth.
United States Patent Publication 20010055336 to Krause, et al. published Dec. 27, 2001 and entitled “Compressed-Video Re-encoder System For Modifying The Compression Ratio Of Digitally Encoded Video Programs” discloses a compressed video decoder/encoder (re-encoder) system for varying the compression ratio of a compressed video program. The composite re-encoder system implements tightly coupled elements for decoding and encoding compressed video data implementing techniques of header forwarding and utilizing an architecture in which a shared motion compensator supports both decoding and encoding operations simultaneously. The re-encoder system may be introduced in a statistical multiplexer for generating a compressed video data stream multiplex suitable for use in cable distribution and other video distribution systems.
United States Patent Publication No. 20020085584 to Itawaki, et al. published Jul. 4, 2002 entitled “Statistical multiplex system, statistical multiplex controller and method of statistical multiplex” discloses a statistical multiplex system, a statistical multiplex controller and a method of statistical multiplex, which can assign bit rates to program data and auxiliary data for purposes of image quality. A statistical multiplex system is provided with: a plurality of image encoders for encoding a plurality of program data; an information encoder for encoding the auxiliary data; a multiplexing apparatus for multiplexing output thereof, and a statistical multiplex controller for controlling each of the image encoders and the information encoder. The statistical multiplex controller is made to set the bit rate to be assigned to the information encoder first, and to assign the remaining bit rates to each of the image encoders.
While the prior art has in general recognized the utility of creating at one video encoding stage information helpful to a next video encoding stage (and communicating such information to the next stage), and the creation of descriptive metadata relating to content, it fails to take into account the fact that at the input to an early (e.g., the first) video compression stage, the highest visual quality content is available. Furthermore, the prior art fails to provide for a suitable method of generating and distributing information related to e.g., the bitrate of the content as a function of time during the content conditioning step described above. Such information could be useful to multiplexers downstream to, inter alia, provide for more efficient and effective program multiplexing and allocation, and improve the visual quality of multiplexed or re-multiplexed video.
Furthermore, the prior art fails to provide a suitable method of multiplexing that avoids computationally intensive techniques to distribute programs in a multiplex, such as by using heuristic parameters such as optimization of visual quality of programs in the multiplex.
Accordingly, it would be most desirable to implement methods and apparatus that generate information regarding bitrates suitable to encode a program, distribute such information within a network and use this information at various locations in the network to provide high visual quality programs to users. Similarly, it is desirable to implement methods and apparatus that provide a multiplexer that uses such information to efficiently create output multiplexes. For the ease of deployment in a content distribution network, such methods and apparatus should also be one that requires minimal changes to existing infrastructure and also is flexible to allow operator to change working parameters to adapt the technique to run-time changes in the network bandwidth utilization.